Model Connection

HAWKI Model Connection Data Flow

This section describes the data flow in HAWKI's AI model connection system, from the moment a request is received in the StreamController->handleAiConnectionRequest method, through the formatting of data for AI models, to how the model responses are processed and formatted for HAWKI.

Overview

HAWKI's AI integration uses a service-based architecture to process requests to various AI models (OpenAI, GWDG, Google). The system is designed to handle both streaming and non-streaming responses, and supports different AI providers with their specific API requirements.

Key Components

The AI connection system consists of several key services:

1. StreamController: Entry point for AI requests handling both direct and group chat interactions
2. AiPayloadFormatterService: Formats user messages into provider-specific payloads
3. ModelConnectionService: Manages the HTTP connections to AI model providers
4. AiResponseFormatterService: Processes and formats responses from AI models
5. ModelUtilityService: Provides utility functions for model configuration and provider detection
6. UsageAnalyzerService: Tracks and records token usage for analytics and billing

Detailed Data Flow

1. Request Reception

The flow begins when a client sends a request to the handleAiConnectionRequest method in the StreamController. The request includes:

• A payload containing the model ID, streaming preference, and messages
• Additional metadata for handling the response (broadcast flags, message IDs, etc.)

$validatedData = $request->validate([
    'payload.model' => 'required|string',
    'payload.stream' => 'required|boolean',
    'payload.messages' => 'required|array',
    'payload.messages.*.role' => 'required|string',
    'payload.messages.*.content' => 'required|array',
    'payload.messages.*.content.text' => 'required|string',
    
    'broadcast' => 'required|boolean',
    'isUpdate' => 'nullable|boolean',
    'messageId' => 'nullable|string',
    'threadIndex' => 'nullable|int', 
    'slug' => 'nullable|string',
    'key' => 'nullable|string',
]);

2. Initial Payload Formatting

The received request is passed to the AiPayloadFormatterService which formats the messages for the appropriate AI provider:

$formattedPayload = $this->payloadFormatter->formatPayload($validatedData['payload']);

The formatPayload method:

1. Identifies the provider based on the model ID
2. Applies provider-specific formatting rules
3. Returns a properly formatted payload that matches the provider's API requirements

Provider-Specific Formatting

For OpenAI/GWDG:

• Handles special cases for models like mixtral-8x7b-instruct and o1
• Extracts the text content from the nested content structure
• Returns a payload with model, messages, and stream parameters

return [
    'model' => $payload['model'],
    'messages' => $formattedMessages,
    'stream' => $payload['stream'],
];

For Google:

• Transforms role names (assistant → model)
• Restructures the content into Google's expected "parts" format
• Returns a payload with model, contents, and stream parameters

return [
    'model' => $payload['model'],
    'contents' => $formattedMessages,
    'stream' => true,
];

3. Request Handling Flow Determination

Based on the broadcast flag, the request flows through one of two paths:

1. Group Chat Path (handleGroupChatRequest): For messages that need to be broadcasted to a room
2. Direct Path: For one-on-one AI conversations

The system also checks if the model supports streaming:

if($formattedPayload['stream'] && $model['streamable']){
    $formattedPayload['stream_options'] = [
        "include_usage"=> true,
    ];
    $this->createStream($formattedPayload);
}
else{
    $data = $this->createRequest($formattedPayload);
    return response()->json($data);
}

4. Connection to AI Model

The ModelConnectionService handles the actual HTTP connection to the AI provider's API:

Non-Streaming Requests

For non-streaming requests, requestToAiModel or requestToGoogle methods:

1. Set up the necessary HTTP headers
2. Retrieve the provider configuration
3. Create a cURL request with the formatted payload
4. Send the request to the provider's API endpoint
5. Return the complete response

$response = $this->modelConnection->requestToAiModel($formattedPayload);

Streaming Requests

For streaming responses, the streamToAiModel method:

1. Sets up SSE (Server-Sent Events) headers
2. Configures a persistent cURL connection
3. Uses a callback function to process each chunk as it arrives
4. Maintains the connection until the response is complete

$this->modelConnection->streamToAiModel($formattedPayload, $onData);

The callback function handles each chunk of data:

$onData = function ($data) use ($user, $avatar_url, $formattedPayload) {
    // Process chunks and send to client
};

5. Response Formatting

The AiResponseFormatterService processes responses from the AI models:

For Non-Streaming Responses

The formatDefaultResponse or formatGoogleResponse methods:

1. Parse the JSON response
2. Extract the content and usage information
3. Return the content and usage as an array

[$content, $usage] = $this->responseFormatter->formatDefaultResponse($response);

For Streaming Responses

The formatDefaultChunk method:

1. Parses each JSON chunk
2. Checks if it's the final chunk
3. Extracts usage information if available
4. Returns the content fragment, completion status, and usage data

[$chunk, $isDone, $usage] = $this->responseFormatter->formatDefaultChunk($chunk);

6. Usage Tracking

The UsageAnalyzerService records token usage for analytics and billing:

$this->usageAnalyzer->submitUsageRecord($usage, 'private', $formattedPayload['model']);

This records:

• Prompt and completion tokens
• Model used
• Type of conversation (private or group)
• User and room information (if applicable)

7. Response Delivery

Finally, the formatted response is delivered to the client:

For Non-Streaming Responses

A complete JSON response is returned:

$messageData = [
    'author' => [
        'username' => $user->username,
        'name' => $user->name,
        'avatar_url' => $avatar_url,
    ],
    'model' => $formattedPayload['model'],
    'isDone' => true,
    'content' => $content,
];
return $messageData;

For Streaming Responses

Each chunk is immediately sent to the client:

echo json_encode($messageData). "\n";

For Group Chats

In group chat mode, the response is:

1. Encrypted with the room's key
2. Stored in the database
3. Broadcasted to all members via Laravel events

$encryptiedData = $cryptoController->encryptWithSymKey($encKey, $content, false);

$message = Message::create([
    'room_id' => $room->id,
    'member_id' => $member->id,
    'message_id' => $nextMessageId,
    'message_role' => 'assistant',
    'model' => $formattedPayload['model'],
    'iv' => $encryptiedData['iv'],
    'tag' => $encryptiedData['tag'],
    'content' => $encryptiedData['ciphertext'],
]);

SendMessage::dispatch($message, $isUpdate)->onQueue('message_broadcast');

Provider-Specific Handling

OpenAI/GWDG Format

Input message format:

{
  "model": "gpt-4o",
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Hello, how are you?"}
  ],
  "stream": true
}

Response format:

{
  "choices": [
    {
      "delta": {
        "content": "I'm doing well, thank you for asking!"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 23,
    "completion_tokens": 12
  }
}

Google Format

Input message format:

{
  "contents": [
    {
      "role": "user",
      "parts": {
        "text": "Hello, how are you?"
      }
    }
  ]
}

Response format:

{
  "candidates": [
    {
      "content": {
        "parts": [
          {
            "text": "I'm doing well, thank you for asking!"
          }
        ]
      }
    }
  ],
  "usageMetadata": {
    "promptTokenCount": 23,
    "candidatesTokenCount": 12
  }
}

Error Handling

The system includes error handling at multiple levels:

1. Request validation in the controller
2. Provider availability checking in the formatter
3. Connection error handling in the model connection service
4. Response parsing error handling in the formatter

When errors occur, they are logged and appropriate error responses are returned to the client.